Systems for parallel processing of datasets with dynamic skew compensation

ABSTRACT

Systems and methods are provided for parallel processing of datasets with dynamic skew compensation. The disclosed systems and methods may increase the efficiency of dataset processing by imposing maximum size limits on parallel processing environment tasks. The disclosed systems and methods may generate a target partition of a variable, a database storing data elements, a cluster that generates one or more output files based on the target partition and the data elements, and a display device that displays analysis results for the target partition using the one or more output files. Generation may comprise creating a calculation partition, mapping data elements according to the calculation partition, and generating the one or more output files based on the mapped data elements. The calculation partition may depend on a target partition and a uniform partition that partitions values based on one or more of statistical measures and pseudorandom functions.

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/271,937, filed on Sep. 21, 2016, which claims priority from U.S.Provisional Patent Application No. 62/221,434, filed on Sep. 21, 2015.The entire contents of these disclosures are incorporated by referencesin the present application.

TECHNICAL FIELD

The disclosed systems provide parallel processing for data analysis. Inparticular, the disclosed systems disclose an improved datasetprocessing system with dynamic skew compensation. The disclosed systemssolve problems related to minimizing processing execution time by, amongother things, efficiently distributing the dataset processing amongnodes in a cluster, and allocating resources.

BACKGROUND

The term “Big Data” typically refers to datasets too large forprocessing in a traditional manner on a single computer. Instead, theanalysis of such datasets may require distributing tasks across multiplecomputing devices, or nodes. These nodes may then efficiently executethe distributed tasks in parallel. But a non-uniform distribution oftasks across nodes may dramatically reduce the efficiency of thisapproach. Such a non-uniform distribution of tasks may arise whenprocessing data with a non-uniform (or skewed) distribution of values.

For example, a dataset of customer accounts may include a skeweddistribution of customer ages, due to the difference in populationbetween different generations. This skewed distribution may inhibitefficient analysis of this dataset. For example, when the customeraccounts are grouped by customer age into decades, the resulting decadegroups will contain unequal numbers of accounts. The time efficiencywith which the analysis of the dataset according to this decade groupingcould be accomplished would be adversely affected by the time requiredto analyze the largest group.

Existing methods may attempt to improve efficiency by monitoring nodesduring execution of an analysis. These methods may rely on detecting andstopping long-running tasks. The stopped tasks may be divided intosmaller tasks and redistributed among the nodes. While such monitoringmay be performed automatically, such automatic methods are experimental,unstable, and difficult to apply to some calculations. Manual methodsnecessitate manually updating software instructions, and are thereforetedious, complicated, and error-prone. Other existing methods simplyiterate trial analyses, updating task allocations until an efficientallocation is discovered. But the duration of each trial analysis mayvary from minutes to hours, rending this approach unpredictable andinefficient. A need therefore exists for improved dynamic skewcompensation for parallel processing of large datasets.

SUMMARY

The disclosed systems determine a uniform distribution of work in eachtask prior to execution, providing more efficient, predictable executionwithout the drawbacks of current methods that may identify long-runningtasks only during or after execution. The disclosed systems do notnecessitate manually updating software instructions, as in manual tasksubdivision. Nor do the disclosed systems risk system instability andjob scheduler perturbations, as in dynamic task subdivision. As anadditional benefit, the envisioned embodiments impose maximum tasksizes, with associated maximum execution times, improving the accuracyand reliability of the job scheduler's management of the parallelcomputing environment during execution of the analysis.

The disclosed systems may determine a uniform partition of data elements(e.g., sensor measurements, log files, streaming data, database rows,records, objects, documents, or other data structures storinginformation) relevant to an analytic objective. As a non-limitingexample, when the analytic object comprises determining financialaccount balances according to selected customer characteristics (e.g.,demographic information, financial information, personal information,etc.), the disclosed systems may determine a uniform partition of dataelements by the selected customer characteristics. This uniformpartition may be based on statistical measures, such as quantiles.Computing these statistical measures may require processing of everyelement of the dataset, but this computation may require seconds tominutes. Accordingly, the efficiency gains from uniformly distributingtasks more than offset the expense of computing the statisticalmeasures.

The disclosed systems may be configured to analyze a dataset accordingto a target partition. A target partition may group data elementsaccording to numerical variable target ranges, categorical-valuedvariables, or categorical-valued variable groups. For example, a targetpartition may group data elements according to the numerical variable“age” into decades, or into more complicated groupings, such as agegroups (e.g., customers younger than 18, 18-24, 25-34, 45-64, andcustomers over 64). As an additional example, a target partition maygroup data elements according the categorical variable “state ofresidence” into geographical regions (e.g., the regions “Northeast,”“Southeast,” “Midwest,” “Central,” “Northwest,” and “Southwest”).

The disclosed systems may determine a calculation partition based on thetarget partition and a uniform partition of the variable. The disclosedsystems may determine a calculation partition for multiple targetpartitions. For instance, the disclosed systems may create a calculationpartition for an analysis grouping financial services customers by bothage ranges and credit risk score ranges. The disclosed systems maycreate a calculation partition grouping data elements across bothnumerical ranges and categorical variable values, or groups of values(e.g., “state of residence” or geographical region). In each case, thedisclosed systems may aggregate data corresponding to subsets of thecalculation partition to generate intermediate values. The disclosedsystems may further aggregate these intermediate values into analysisresults satisfying the analytic objective. As used herein, results maycomprise scalar values, arrays, lists, dictionaries, records, objects,functions, or other datatypes known to one of skill in the art.

The disclosed systems may be implemented using a parallel computingenvironment, such as the MapReduce architecture described in “MapReduce:Simplified Data Processing on Large Clusters,” by Jeffrey Dean andSanjay Ghemawat, or the Spark architecture described in “Spark: ClusterComputing with Working Sets,” by Matei Zaharia, Mosharaf Chowdhury,Michael J. Franklin, Scott Shenker, and Ion Stoica, each of which isincorporated herein by reference in its entirety. In such anarchitecture (e.g., Apache™ Hadoop® (see http://hadoop.apache.org/) orthe like), the disclosed systems may comprise mappers and reducers. Inthe context of MapReduce, map functions perform simple computations andfiltering operations on discrete elements of the input dataset.Typically, the input to a mapper is a series of key-value pairs, whichare processed individually and result in an output of zero or morekey-value pairs. Reduce functions perform a summary operation on theoutput directly or indirectly generated by mappers. One reducer worksonce for each unique key generated from the previous operation in thesorted order. For each key, reducers will iterate through all the valuesassociated with each key and generate zero or more outputs. Thedisclosed system relies on the map and reduce, or similar, capabilitiesin the resident parallel computing environment and improves executionefficiency by configuring map and reduce, or similar, functions toevenly distribute work among the mappers and reducers.

The disclosed system may include calculation partitions that define thekeys by which the mappers evenly group data elements for aggregation bythe reducers. The reducers may then compute aggregate results for eachgroup of mapped data elements. In some embodiments, the disclosedsystems may create two phases of reducers. A first phase of reducers maydetermine intermediate values according to the calculation partition. Asecond phase of reducers may determine analysis results based on theintermediate values. In certain embodiments, another computing devicemay determine analysis results based on intermediate results provided bythe parallel computing environment.

The disclosed systems are not limited to a specific parallelizationtechnology, job scheduler (e.g., Apache™ Hadoop® YARN (Yet AnotherResource Negotiator, seehttp://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html)or Apache™ Mesos (see http://mesos.apache.org/)), programming language,parallel computing environment, or parallel computing environmentcommunications protocol. For example, the disclosed systems may beimplemented in scientific computing clusters, databases, cloud-basedcomputing environments, and ad-hoc parallel computing environments(e.g., SETI@home (see http://setiathome.berkeley.edu/) or the like).

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive of the disclosed embodiments, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings are not necessarily to scale or exhaustive.Instead, emphasis is generally placed upon illustrating the principlesof the inventions described herein. These drawings, which areincorporated in and constitute a part of this specification, illustrateseveral embodiments consistent with the disclosure and, together withthe detailed description, serve to explain the principles of thedisclosure. In the drawings:

FIG. 1 depicts an exemplary skewed cumulative distribution functionconsistent with disclosed embodiments.

FIG. 2 depicts a schematic illustrating an exemplary system for parallelprocessing datasets with dynamic skew compensation.

FIGS. 3A-3C depict schematics illustrating exemplary calculationpartitions.

FIG. 4 depicts schematics illustrating exemplary datasets generated by asystem for parallel processing datasets with dynamic skew compensation.

FIG. 5 depicts a flowchart illustrating exemplary operations forprocessing datasets with dynamic skew compensation.

FIG. 6 depicts a schematic illustrating an exemplary component of adataset processing system.

DETAILED DESCRIPTION

Reference will now be made in detail to the disclosed embodiments,examples of which are illustrated in the accompanying drawings. Whereverconvenient, the same reference numbers will be used throughout thedrawings to refer to the same or like parts.

FIG. 1 depicts an exemplary skewed function 100 of a variable,consistent with disclosed embodiments. In some embodiments, the skewedfunction 100 may be a cumulative distribution function. Skewed function100 describes a continuously valued numeric variable (such as afinancial account balance), but one of skill in the art would recognizethe applicability of the following disclosure to discrete-valued numericvariables (such as the age of an account holder in years). Consistentwith disclosed embodiments, the disclosed systems may be configured toanalyze data elements according to target partition 101. The number ofdata elements, however, may differ between subsets of target partition101. In contrast, uniform partition 105 may divide the data elementsinto similarly sized groups, but these groups may not correspond totarget ranges of target partition 101. As a non-limiting example, asshown in FIG. 1, target partition 101 defines six target ranges, whileuniform partition 105 defines five different similarly sized groups ofdata elements.

The disclosed systems may be configured to determine a uniform partition105 that divides data elements into similarly sized groups. As usedherein, the number of data elements the similarly sized groups ofuniform partition 105 may differ by less than 50%. As a non-limitingexample, when the smallest group of uniform partition 105 comprises 100data elements, the largest group of uniform partition 105 may comprisefewer than 150 data elements. As a further non-limiting example, thesimilarly sized groups of uniform partition 105 may differ may differ byless than 10%.

The disclosed systems may be configured to determine a uniform partition105 based on statistical measures 103. The particular number of subsetsin uniform partition 105 may depend on the skewness of the data (e.g.,non-uniform distribution of certain values of the data) and the numberof nodes available in the parallel processing environment. It is commonfor the number of data elements in a dataset to be skewed, or have adifferent count of values in each index or key that a reducer will needto operate on. For example, the number of null or 0 values may be quitehigh for a key that is defined by a specific field in a dataset,creating significant skew, and disproportionate work for the reducerassociated with the null or 0 key. For clarity, FIG. 1 displays a smallnumber of uniform partitions (5) and a small number of target partitions(6). One of skill in the art would appreciate that these numbers are notintended to be limiting. For example, a greater number of groups of dataelements and of target ranges may be used in practice. Furthermore, oneof skill in the art would appreciate that uniform partition 105 mayinclude far more groups of data elements than target partition 101includes target ranges.

Consistent with disclosed embodiments, statistical measures 103 may be afunction of the values of the data elements. In some embodiments,statistical measures 103 may depend on a distribution function of thevalues of the data elements, such as, but not limited to, a probabilitydistribution function, cumulative distribution function, or betacumulative distribution function. In certain embodiments, statisticalmeasures 103 may be quantiles or estimated quantiles. One of skill inthe art would be familiar with systems and methods for determining bothexact and estimated quantiles. As a non-limiting example, methods forestimating quantiles are disclosed in “A new distribution-free quantileEstimator” by Frank E. Harrell and C. E. Davis; “Sample Quantiles inStatistical Packages” by Hyndman and Fan; “Approximating quantiles invery large datasets,” by Reza Hosseini; “A One-Pass Algorithm forAccurately Estimating Quantiles for Disk-Resident Data” by KhaledAlsabti, Sanjay Ranka, and Vineet Singh; and “Effective Computation ofBiased Quantiles over Data Streams” by Graham Cormode, Flip Korn, S.Muthukrishnany, and Divesh Srivastava, each of which is incorporatedherein by reference in its entirety. Furthermore, as would be recognizedby one of skill in the art, the envisioned embodiments are not intendedto be limited to particular methods of determining exact and estimatedquantiles. As a non-limiting example, the quantiles or estimatedquantiles may range from deciles to percentiles. For example, thequantiles or estimated quantiles may be twentiles.

As discussed in greater detail below, the disclosed systems may derive acalculation partition for one or more variables. This calculationpartition may be used by the disclosed systems for processing dataelements to generate results. In certain aspects, the calculationpartition may comprise partitions for each of the variables. In someembodiments, these partitions may be functions of a target partition(e.g., target partition 101) and a uniform partition (e.g., uniformpartition 105). For example, in some aspects, these partitions may becombination partitions. As described below in greater detail,combination partitions may comprise the union of a target partition(e.g., target partition 101) and a uniform partition (e.g., uniformpartition 105). For example, a combination partition for thecontinuously valued numeric variable shown in FIG. 1 may comprise theunion of target partition 101 and uniform partition 105.

Analysis information may comprise data and/or instructions stored in anon-transitory memory, consistent with disclosed embodiments. In someembodiments, analysis information may comprise one or more of a mappingfunction, a reduction function, target partitions (e.g., targetpartition 101), uniform partitions (e.g., uniform partitions 105) andstatistical measures (e.g., statistical measures 103). In some aspects,the mapping function may implement the target partition. In certainaspects, the mapping function may implement the calculation partition.In various aspects, the mapping function may comprise data and/orinstructions stored in a non-transitory memory. In certain aspects, thereduction function may specify operations to be performed on one or morevariables of the data elements. These operations may include aggregatinggroups of values. As an non-limiting example, a reduction function mayconfigure dataset processing system 200 to sum a number of financialservices accounts and average the values of these accounts, and amapping function may configure dataset processing system 200 to generatethese sums and averages by, as non-limiting examples, an account holderstate of residence, a delinquency status of the account, and decade ofaccount holder age.

FIG. 2 depicts a schematic illustrating an exemplary system (datasetprocessing system 200) for parallel processing datasets with dynamicskew compensation, consistent with disclosed embodiments. In someembodiments, dataset processing system 200 may be configured to impose amaximum group size by generating a calculation partition combining auniform partition (e.g., uniform partition 105) with a target partition(e.g., target partition 101). Dataset processing system 200 may beconfigured to process data elements according to this calculationpartition, and further process these secondary values to return analysisresults aggregated into the target ranges of the target partition (e.g.,target partition 101).

Dataset processing system 200 may comprise one or more of user device201, datasource 203, cluster 205, and display device 209, consistentwith disclosed embodiments. In certain aspects, cluster 205 may beconfigured to generate output data 207 using data elements received fromdatasource 203 according to analysis information provided by user device201. Display device 209 may be configured to receive output data 207 anddisplay analysis results according to the analysis information providedby user device 201. In various aspects, user device 201, datasource 203,cluster 205, and display device 209 may communicate over a network. Insome aspects, the network may be any type of network (includinginfrastructure) that provides communications, exchanges information,and/or facilitates the exchange of information between the nodes. As anon-limiting example, the network may comprise a Local Area Network.

User device 201 may be configured to provide analysis information toother components of dataset processing system 200, consistent withdisclosed embodiments. User device 201 may include, but is not limitedto, a general purpose computer, computer cluster, terminal, mainframe,mobile computing device, or other computing device capable of providinganalysis information to other components of dataset processing system200. For example, a general purpose computer may include, but is notlimited to, a desktop, workstation, or all-in-one system. As anadditional example, a mobile computing device may include, but is notlimited to, a cell phone, smart phone, personal digital assistant,tablet, or laptop. In some embodiments, user device 201 may be a clientdevice of another component of dataset processing system 200. Anon-limiting example of such a computing device is provided below inFIG. 6. In some aspects, a user (not shown) may operate user device 201,or direct operation of user device 201.

User device 201 may be configured to generate the uniform partitionbased on statistical measures (e.g., statistical measures 103),consistent with disclosed embodiments. In certain aspects, user device201 may be configured to receive these statistical measures from anothercomponent of dataset processing system 200.

User device 201 may be configured to assign tasks to cluster 205. Forexample, user device 201 may be configured to assign mapping and/orreducing tasks to cluster 205. In some aspects, user device 201 may beconfigured to assign tasks to cluster 205 directly, by assigning tasksto worker nodes of cluster 205. In certain aspects, user device 201 maybe configured to assign tasks to cluster 205 indirectly, by interactingwith controller 211, as described below.

Datasource 203 may be configured to provide information to components ofdataset processing system 200, consistent with disclosed embodiments.For example, datasource 203 may comprise a network socket. As a furtherexample, datasource 203 may comprise a source of messages in apublication and subscription framework (e.g., Apache™ Kafka (seehttp://kafka.apache.org/)). Components of dataset processing system 200may be configured to receive data elements through the network socket,or from the message source. As an additional example, datasource 203 maycomprise a data stream, such as computer network traffic, financialtransactions, or sensor data. In further examples the datasource maycomprise data produced by another computing device, such as a computingcluster.

In some embodiments, datasource 203 may comprise a storage device thatstores information for access and management by components of datasetprocessing system 200. In some aspects, the storage device may beimplemented in a non-transitory memory, such as one or more memorybuffers, a solid state memory, an optical disk memory, or a magneticdisk memory. In various aspects, cluster 205 may implement the storagedevice. In some aspects, the storage device may comprise a distributeddata storage system, such as Apache™ HDFS™ (Hadoop® Distributed FileSystem, see https://hadoop.apache.org/docs/r1.2.1/hdfs_design.html),Apache™ Cassandra™ (see http://cassandra.apache.org/), Apache™ HBase™(Hadoop® Database, see https://hbase.apache.org/), or Amazon™ S3(Amazon™ Simple Storage Service, see https://aws.amazon.com/s3/) Asshown in FIG. 2, cluster 205 may comprise datasource 203. Alternatively,datasource 203 may be distinct from cluster 205 (not shown), and cluster205 and datasource 203 may be connected by a network, as describedabove.

In some embodiments, datasource 203 may be implemented as a hierarchicaldatabase, relational database, object-oriented database,document-oriented database, graph-oriented database, key-value database,or any combination thereof. One of skill in the art would recognize manysuitable implementations of datasource 203, and the envisionedembodiments are not intended to be limited to a particularimplementation. Datasource 203 may be configured to provide dataelements to cluster 205, consistent with disclosed embodiments. In someaspects, datasource 203 may provide data elements to cluster 205 inresponse to an indication from one or more of cluster 205 and userdevice 201.

In some embodiments, cluster 205 may comprise a collection of nodesconnected over a network. In some aspects, the nodes may comprisecomputing devices, such as servers, workstations, desktops, graphicscards, videogame systems, embedded systems, etc. A non-limiting exampleof such a computing device is provided below in FIG. 6. In certainaspects, the network may be any type of network (includinginfrastructure) that provides communications, exchanges information,and/or facilitates the exchange of information between the nodes. As anon-limiting example, the network may comprise a Local Area Network. Incertain embodiments, one or more nodes may comprise virtual nodes. Insome embodiments, cluster 205 may exceed 100 nodes.

In some embodiments, cluster 205 may include a workstation comprisingnodes. In certain aspects, the workstation may comprise multiplegraphics processing units. The nodes may comprise the graphicsprocessing units, or components of the graphics processing units, suchas cores. The workstation may comprise non-transitory individualmemories for the graphics processing units. As a non-limiting example,each graphics processing unit may use an individual memory. In thisnon-limiting example, the graphics processing units may not beconfigured to share the individual memories.

Cluster 205 may comprise one or more nodes configured as a worker node,consistent with disclosed embodiments. In various aspects, cluster 205may be configured to assign resources to worker nodes. As a non-limitingexample, cluster 205 may assign to worker nodes memory resources, suchas a total amount of memory, or physical or virtual memory locations, orother memory resources as would be known by one of skill in the art. Asanother non-limiting example, cluster 205 may assign to worker nodesprocessor resources, such as cores, threads, physical or virtualprocessors, or other processor resources as would be known by one ofskill in the art.

Cluster 205 may be configured to assign a number of worker nodes asmapper(s) 213 and/or reducer(s) 217, as discussed in greater detailbelow, consistent with disclosed embodiments. In some embodiments,cluster 205 may comprise a controller, such as controller 211, thatconfigures worker nodes with mapping or reduction tasks. In variousaspects, cluster 205 may determine the number of worker nodes assignedas mapper(s) 213. In some aspects, the number of number of worker nodesassigned as mapper(s) 213 and/or reducer(s) 217 may be determinedmanually or at least in part automatically. In certain aspects, theresources assigned to mapper(s) 213 and/or reducer(s) 217 may bedetermined manually or at least in part automatically. In some aspects,cluster 205 may be configured to temporarily or dynamically assignworker nodes as mapper(s) 213 and/or reducer(s) 217.

Consistent with disclosed embodiments, as described below with regard toFIG. 4, cluster 205 may be configured to process data elements accordingto analysis information received from user device 201. Cluster 205 maybe configured to store the received analysis information as data and/orinstructions in a non-transitory memory. In some embodiments, analysisinformation may comprise a mapping function implementing a calculationpartition. Cluster 205 may be configured to process data elementsaccording to the calculation partition. In certain embodiments, analysisinformation may comprise a mapping function implementing a targetpartition. Cluster 205 may be configured to augment the received mappingfunction to implement a calculation partition based on the receivedanalysis information. Depending on the received analysis information,augmenting the mapping function may comprise determining one or more ofthe calculation partition, a uniform partition, and statistical measuresfor one or more variables. For example, cluster 205 may be configured todetermine the calculation partition based on the received targetpartition. As an additional example, cluster 205 may be configured todetermine the calculation partition based on a received uniformpartition. As an additional example, cluster 205 may be configured toderive a uniform partition based on the received analysis information.For example cluster 205 may be configured to determine the uniformpartition based on received statistical measures. As a further example,cluster 205 may be configured to determine statistical measures based ondata elements received from datasource 203. In various embodiments,cluster 205 may be configured to receive a mapping function implementingthe calculation partition.

In certain embodiments, cluster 205 may be configured to receive fromuser device 201 a reduction function for processing groups of values.The reduction function may be used to compute and extract certainfeatures of the received data. As a non-limiting example, the reductionfunction may sum a number of financial services accounts and average thevalues of these accounts. Other examples include calculating the medianor finding the top 10 values of any numeric field, such as financialservices account balances. Other operations are possible as well. Thereceived reduction function may comprise data and/or instructions storedin a non-transitory memory. Cluster 205 may be configured to generateoutput data 207 using the reduction function.

Output data 207 may comprise data or instructions derived from dataelements, consistent with disclosed embodiments. In some embodiments,output data 207 may be stored in at least one non-transitory memory. Forexample, output data 207 may be stored in one or more memory buffers, asolid state memory, an optical disk memory, or a magnetic disk memory.In various aspects, one or more of cluster 205, user device 201, and/ordisplay device 209 may be configured to store output data 207. Incertain aspects, output data 207 may be stored on one or more remotenon-transitory memories, according to systems known to one of skill inthe art. In some aspects, output data 207 may be stored using adistributed storage system, such as Apache™ HDFS™, Apache™ Cassandra™,Apache™ HBase™, or Amazon™ S3. As would be recognized by one of skill inthe art, the particular storage location of output data 207 is notintended to be limiting. In some embodiments, output data 207 may bestreamed from a network socket, or provided as messages in a publicationand subscription framework.

Output data 207 may include one or more key-value pairs, as described ingreater detail below with regard to FIG. 4, consistent with disclosedembodiments. In some aspects, the key may correspond to a subset of acalculation partition. In various aspects, the key may correspond to asubset of the product of one or more target partitions and one or morecategorical variable(s). In certain aspects, the value may be the resultof performing an aggregation function on one or more values associatedwith the key. Output data 207 may be formatted as streamed dataelements, text files, database files, or other data formats known to oneof skill in the art. One of skill in the art would recognize manysuitable formats for output data 207, and the envisioned embodiments arenot intended to be limited to a particular format.

Display device 209 may be configured to receive data and/or instructionsfrom other components of dataset processing system 200, consistent withdisclosed embodiments. In some aspects, display device 209 may beconfigured to retrieve or receive output data 207. For example, displaydevice 209 may be configured to retrieve output data 207 from acomponent of dataset processing system 200, such as cluster 205,datasource 203, user device 201, and/or one or more remotenon-transitory memories. In various aspects, display device 209 may beconfigured to receive output data 207 streamed from a network socket, orprovided as messages in a publication and subscription framework. Insome embodiments, display device 209 may be configured to re-apply thereduction function, or another function, to output data 207 to generateanalysis results, as described in detail below.

Display device 209 may be configured to provide analysis results.Providing analysis results may comprise displaying the analysis results,printing the analysis results, or storing them on a non-transitorycomputer readable medium. As would be recognized by one of skill in theart, the particular method of providing the analysis results is notintended to be limiting.

Display device 209 may include, but is not limited to, a general purposecomputer, computer cluster, terminal, mainframe, or mobile computingdevice capable of receiving and displaying data and/or instructions. Forexample, a general purpose computer may include, but is not limited to,a desktop, workstation, or all-in-one system. As an additional example,a mobile computing device may include, but is not limited to, a cellphone, smart phone, personal digital assistant, tablet, or laptop. Insome embodiments, display device 209 may be a client device of anothercomponent of dataset processing system 200. In certain embodiments,display device 209 and user device 201 may be the same device. In someembodiments, a user (not shown) may operate display device 209, ordirect operation of display device 209.

Controller 211 may comprise one or more nodes of cluster 205 configuredwith data and instructions for managing the operations of datasetprocessing system 200, consistent with disclosed embodiments. In someembodiments, controller 211 may be configured to assign nodes of cluster205 as worker nodes. In certain embodiments, controller 211 may beconfigured to assign worker nodes as mapper(s) 213. As an additionalexample, controller 211 may be configured to assign worker nodes asreducer(s) 217. In some embodiments, controller 211 may also beconfigured to track the status of mapper(s) 213 and reducer(s) 217,assigning and reassigning tasks (such as mapping and reduction) toworker nodes. In some embodiments, controller 211 may be configured toprovide a mapping function to mapper(s) 213 and/or a reduction functionto reducer(s) 217. In some embodiments, controller 211 may be configuredto enable reducer(s) 217 to access intermediate data 215. For example,controller 211 may be configured to store locations of intermediate data215, and provide these locations to reducer(s) 217. In certainembodiments, controller 211 may be configured to shuffle, or route,intermediate data from mapper(s) 213 to reducer(s) 217.

Mapper(s) 213 may comprise one or more nodes of cluster 205 configuredwith data and instructions for processing data elements. Consistent withdisclosed embodiments, mapper(s) 213 may be configured to receive dataelements and yield processed data elements according to a mappingfunction. As a non-limiting example, a data element may be associatedwith a key value according to the mapping function. A mapper processinga data element according to the mapping function may receive the dataelement and yield a processed data element. In some embodiments, theprocessed data element may comprise the key value. For example, a mapperprocessing words according to a mapping function that maps words to thenumber of letters in the word may receive the sentence “Midway on ourlife's journey, I found myself in dark woods, the right road lost” andyield {6, 2, 3, 5, 7, 1, 5, 6, 2, 4, 5, 3, 5, 4, 4}. In someembodiments, the processed data element may comprise the key-valuetogether with at least a portion of the received data element. Forexample, a mapper processing {age, account balance} tuples according toa mapping function that maps ages to decades may receive the dataelements {{35, $200}, {23, $1,000}, {54, $20,000}} and yield {{3, $200},{2, $1,000}, {5, $20,000}}. In some embodiments, the processed dataelement may comprise the key-value together with a function of thereceived data element. In some embodiments, the mapping function may bereceived from controller 211. In certain embodiments, mapper(s) 213 maybe configured to receive data elements from datasource 203. In someembodiments, mapper(s) 213 may generate intermediate data 215. Incertain aspects, each of mapper(s) 213 may correspond to a portion ofintermediate data 215.

Intermediate data 215 may comprise processed data elements generated bymapper(s) 213, consistent with disclosed embodiments. Intermediate data215 may be stored in at least one non-transitory memory. For example,intermediate data 215 may be stored in one or more memory buffers, asolid state memory, an optical disk memory, or a magnetic disk memory.In various aspects, one or more of cluster 205, user device 201, and/ordisplay device 209 may be configured to store intermediate data 215. Incertain aspects, intermediate data 215 may be stored on one or moreremote non-transitory memories, according to systems known to one ofskill in the art. In some aspects, intermediate data 215 may be storedusing a distributed storage system, such as Apache™ HDFS™, Apache™Cassandra™, Apache™ HBase™, or Amazon™ S3. As would be recognized by oneof skill in the art, the particular storage location of intermediatedata 215 is not intended to be limiting.

In certain embodiments, as described in greater detail below with regardto FIG. 4, intermediate data 215 may be associated with a calculationpartition. In certain aspects, each of intermediate data 215 may includeone or more key-value pairs. In some aspects, the key may correspond toa subset of a calculation partition. In various aspects, the key maycorrespond to a subset of the product of one or more target partitions,and/or one or more categorical variable(s). In certain aspects, thevalue may be derived from a data element. As a non-limiting example,when the data element is a row of a relational database, the value maycomprise one or more entries in the row. As a further non-limitingexample, when the data element is a document, the value may comprise atoken parsed from the document. One of skill in the art would recognizemany suitable formats for intermediate data 215, and the envisionedembodiments are not intended to be limited to a particular format.

Reducer(s) 217 may comprise one or more nodes of cluster 205 configuredwith data and instructions for processing intermediate data 215. In someembodiments, controller 211 may configure reducer(s) 217 to retrievestored intermediate data 215. In certain embodiments, controller 211 maybe configured to shuffle, or route, intermediate data 215 to reducer(s)217.

In some embodiments, reducer(s) 217 may be configured to derivesecondary values and/or analysis results based on received analysisinformation. For example, reducer(s) 217 may be configured to generategroups of intermediate data 215 from corresponding subsets of acalculation partition. In certain aspects, each group may be generatedby one of reducer(s) 217. In various aspects, reducer(s) 217 mayconfigured to apply a reduction function to each group of intermediatedata to generate secondary values. In some aspects, each secondary valuemay be generated by one of reducer(s) 217 and associated with a subsetof the calculation partition. In some embodiments, output data 207 maycomprise the secondary values.

As an additional example, reducer(s) 217 may be configured to generategroups of secondary values corresponding to subsets of a targetpartition. In certain aspects, reducer(s) 217 may be configured to applya function to each group of secondary values to generate analysisresults. For example, reducer(s) 217 may be configured to re-apply thereduction function, or another function. In some aspects, each analysisresult may be generated by one of reducer(s) 217 and associated with asubset of the target partition. In some embodiments, output data 207 maycomprise the analysis results.

FIGS. 3A-3C depict schematics illustrating exemplary calculationpartitions. Dataset 301, as depicted in each of FIGS. 3A-3C, graphicallyindicates the set of data elements received from datasource 203. FIG. 3Adepicts a schematic illustrating an exemplary calculation partitioncomprising a combination partition for a numeric variable. Thiscombination partition comprises the union of target partition 302 anduniform partition 304. In some embodiments, target partition 302 maycomprise data or instructions for dividing dataset 301 into subsets. Incertain aspects, these subsets may be defined by boundary values (e.g.,T₁, T₂, T₃, T₄, and T₅). In various embodiments, uniform partition 304may comprise data or instructions for dividing dataset 301 into subsetscontaining similar numbers of data elements. In certain aspects, thesesubsets may be defined by boundary values (e.g., Ux₁₂, Ux₂₃, Ux₃₄, andUx₄₅). The boundary values of uniform partition 304 may depend onstatistical measures (e.g., statistical measure 103). In some aspects,target partition 302 and uniform partition 304 may contain subsets withcommon boundary values (e.g., Ux₄₅ may equal T₃).

Consistent with disclosed embodiments, the boundary values of thecombination partition may be the union of the boundary values of targetpartition 302 and uniform partition 304 (e.g., T₁, Ux₁₂, Ux₂₃, T₂, Ux₃₄,T₃, T₄, T₅ or equivalently T₁, Ux₁₂, Ux₂₃, T₂, Ux₃₄, Ux₄₅, T₄, T₅). Incertain aspects, the calculation partition may divide the set of dataelements received from datasource 203 into subsets 308 (e.g., r₉). Insome embodiments, the mapping function may define keys 309 associatedwith each of subsets 308. In certain embodiments, a mapping functionimplementing the calculation partition may be arranged to configuremapper(s) 213 to yield intermediate data 215 comprising keys 309according to a mapping function implementing the calculation partition.

FIG. 3B depicts a schematic illustrating an exemplary calculationpartition comprising two partitions, a combination partition for anumeric variable and a categorical partition. The combination partitionfor a numeric variable comprises the union of target partition 312 anduniform partition 314. As discussed above with regards to FIG. 3A,uniform partition 314 of the first numeric variable may comprise data orinstructions for dividing dataset 301 into value ranges containingsimilar numbers of data elements. Likewise, target partition 312 of thefirst numeric variable may comprise data or instructions for dividingdataset 301 into target ranges. Consistent with disclosed embodiments,the boundary values of the combination partition may be the union of theboundary values of target partition 312 and uniform partition 314 (e.g.,T₁, Ux₁₂, Ux₂₃, T₂, Ux₃₄, T₃, T₄, T₅ or equivalently T₁, Ux₁₂, Ux₂₃, T₂,Ux₃₄, Ux₄₅, T₄, T₅). The partition for a categorical variable, targetpartition 315, may comprise data or instructions for configuring datasetprocessing system 200 to divide and aggregate data elements according tothe values of the categorical variable (e.g., C₀ and C₁) and/or groupsof values of the categorical variable.

In certain aspects, the calculation partition may divide the set of dataelements received from datasource 203 into subsets 318 (e.g., r₁₂).These subsets may correspond to the combinations of ranges, categoricalvalues, or categorical groups defined by the combination partition forthe numeric variable and the partition for the categorical variable(e.g., r₁₂ corresponds to the categorical variable C₁ and the numericalrange defined by the boundary values Ux₃₄ and T₃). In some embodiments,the mapping function may define keys 319 associated with each of subsets318. In certain embodiments, the keys may be based on values from arange of values related to size of the available cluster resources. Incertain embodiments, a mapping function implementing the calculationpartition may be arranged to configure mapper(s) 213 to yieldintermediate data 215 comprising keys 319 according to a mappingfunction implementing the calculation partition.

In certain embodiments, cluster 205 may be configured to distribute atleast one task among multiple workers. In certain aspects, the at leastone task may be distributed among multiple workers by associatingmultiple keys with at least one of subsets 318. For example cluster 205may be configured to create additional unique keys for one of subsets318. In certain aspects, one or more of subsets 318 defined by a valueof a categorical variable (e.g., C₁) may be associated with multiplekeys. For example, the mapping function may define sets of three keysfor each of subsets r₃, r₆, r₉, r₁₂, r₁₅, and r₁₈. As an additionalexample, mapper(s) 213 may be configured to yield an intermediate datumcomprising one of the three keys for each data element in subset r₃,according to the mapping function. In some aspects, in this manner, theuniform partition may be based on a random or pseudorandom value. Forexample, mapper(s) 213 may be configured to generate a random orpseudorandom value. In some aspects, this generated value may comprisethe particular key. In some aspects, mapper(s) 213 may be configured tochoose the particular key in the intermediate datum based on thisgenerated value. In certain aspects, in this manner, the uniformpartition may be based on a range of values related to the size of theavailable cluster resources. In certain aspects, in this manner, theuniform partition may be based on a range of values selected to optimizesystem processing speed. This selection may be based on one or more ofempirical trials, simulations, and theoretical modeling, according tomethods known to one of skill in the art. The number of additionalunique keys may be predetermined, or may be dynamically determinedduring processing of the data elements. The value of the additionalunique keys in each set may be chosen to indicate membership of the set.For example, a first set of keys associated with a first subset may havekey values {1-1, 1-2, 1-3} and a second set of keys associated with asecond subset may have key values {2-1, 2-2, 2-3}. One of skill in theart would appreciate that a wide range of values may be used and theabove examples are not intended to be limiting. In this manner, maximumgroup sizes may be established for subsets defined at least in part bycategorical variables. For example, when the categorical variable“geographic region” includes the categories {“northwest” “southwest”“midwest” “central” “northeast” “southeast”}, the number of individualsin the northeast category may be far larger than the number ofindividuals in other categories. By associating multiple keys withsubsets defined at least in part by the category “midwest” of thecategorical variable “geographic region,” more balanced maximum groupsizes may be established. In some embodiments, the number of keys for acategory of the categorical variable may be a function of the number ofitems in the category and the number of items in the smallest categoryof the categorical variable. For example, the number of keys may be thequotient of the number of items in the category divided by the number ofitems in the smallest category.

FIG. 3C depicts a schematic illustrating an exemplary calculationpartition comprising two partitions, a first combination partition for afirst numeric variable and a second combination partition for a secondnumeric variable. Both combination partitions comprise a union of targetpartitions and uniform partitions. As discussed above with regards toFIGS. 3A and 3B, uniform partition 324 of a first numeric variable maycomprise data or instructions for dividing dataset 301 into value rangescontaining similar numbers of data elements. Likewise, target partition322 of the first numeric variable may comprise data or instructions fordividing dataset 301 into target ranges. Consistent with disclosedembodiments, the boundary values of the first combination partition maybe the union of the boundary values of target partition 322 and uniformpartition 324 (e.g., Tx₁, Ux₁₂, Tx₂, Ux₂₃, Tx₃).

Similarly, uniform partition 327 of a second numeric variable maycomprise data or instructions for dividing dataset 301 into value rangescontaining similar numbers of data elements. In some embodiments, targetpartition 325 of the second numeric variable may comprise data orinstructions for dividing dataset 301 into target ranges. Consistentwith disclosed embodiments, the boundary values of the first combinationpartition may be the union of the boundary values of target partition325 and uniform partition 327 (e.g., Uy₁₂, Ty₁).

In certain embodiments, the calculation partition may divide the set ofdata elements received from datasource 203 into subsets 328 (e.g., r₁₇).These subsets may correspond to the combinations of ranges, categoricalvalues, or categorical groups defined by the first and secondcombination partitions (e.g., r₁₇ corresponds to the numerical rangedefined by the boundary values Uy₁₂ and Ty₁ and the numerical rangedefined by the boundary value Tx₃). In some embodiments, the mappingfunction may define keys 329 associated with each of subsets 328. Incertain embodiments, a mapping function implementing the calculationpartition may be arranged to configure mapper(s) 213 to yieldintermediate data 215 comprising keys 329 according to a mappingfunction implementing the calculation partition.

As would be recognized by one of skill in the art, the exemplarycalculation partitions disclosed in FIGS. 3A-3C may be generalized toaddress additional numeric and/or categorical variables. As a furthernon-limiting example, an exemplary calculation partition may comprisethree partitions: a first combination partition for a first numericvariable, a second combination partition for a second numeric variable,and a third partition for a categorical variable. Consistent withdisclosed embodiments, such an exemplary calculation partition maydivide the set of data elements received from datasource 203 intosubsets. This division may be performed according to the operationsdescribed above with regards to FIGS. 3B and 3C. Accordingly, thesystems and methods disclosed herein are not intended to be limited tothe exemplary partitions of FIGS. 3A-3C.

FIG. 4 depicts schematics illustrating exemplary datasets generated by asystem for parallel processing datasets with dynamic skew compensation.In some embodiments, mapper(s) 213 may be configured to generateintermediate data 401. In certain embodiments, reducer(s) 217 may beconfigured to generate secondary values 403 from intermediate data 401.In certain aspects, reducer(s) 217 may be configured to store or streamsecondary values 403 as output data 207. In certain embodiments,reducer(s) 217 may be configured to generate analysis results 405 fromsecondary values 403. In various aspects, reducer(s) 217 may beconfigured to store or stream analysis results 405 as output data 207.In some embodiments, display device 209 may be configured to generateanalysis results 405 from secondary values 403.

Intermediate data 401 may contain processed data elements associatingresults with keys, consistent with disclosed embodiments. In someaspects, the keys may correspond to subsets of the calculation partition(e.g., one of keys 309, 319, or 329). A mapping function may define thisassociation. In certain aspects, a result may comprise at least some ofa data element. For example, when the data element is a row in arelational database, the result may comprise at least some of theelements of the row. In some aspects, the result may depend on the dataelement. In certain aspects, the result may depend on the mappingfunction. For example, the mapping function may configure mapper(s) 213to perform trivially parallelizable calculations on data elements whengenerating intermediate data 401, as would be recognized by one of skillin the art.

Secondary values 403 may comprise keys associated with results,consistent with disclosed embodiments. In some aspects, the keys may beassociated with corresponding subsets of the calculation partition(e.g., one of keys 309, 319, or 329), as described above with regards toFIGS. 3A-3C. In certain aspects, reducer(s) 217 may apply the reductionfunction to groups of intermediate data 401 associated with each key togenerate secondary values 403.

Analysis results 405 may comprise entries associating results withsubsets of one or more target partitions. In certain aspects, analysisresults 405 may comprise entries for every combination of subsets of theone or more target partitions. As shown with respect to FIG. 3B andsecondary values 403, multiple subsets of the calculation partition maycorrespond to a single subset of the target partition. For example,target partition 312 and target partition 315 together define a subsetbounded by values T₁ and T₂ and the categorical value C₀. Subsets r₃,r₅, and r₇ of the calculation partition may correspond to this subset ofthe target partitions. One or more of reducer(s) 217 and display device209 may apply a function to results b₃, b₅, and b₇ associated with r₃,r₅, and r₇ to generate a value c₁ for this subset of the targetpartitions. This function may be the reduction function, or anotherfunction. Values c₂, c₃, and c₄ of analysis results 405 may be similarlydetermined.

FIG. 5 depicts a flowchart illustrating exemplary operations forprocessing datasets with dynamic skew compensation, consistent withdisclosed embodiments. In step 501, the dataset processing system 200may be configured to create a calculation partition. In someembodiments, the calculation partition may comprise one or morecombination partitions. As described above, a combination partitions maycomprise the union of a target partition (e.g., target partition 101)and a uniform partition (e.g., uniform partition 105). The uniformpartition may be derived from statistical measures (e.g., statisticalmeasures 103). The statistical measures may be quantiles.

User device 201 may be configured to create the calculation partition,consistent with disclosed embodiments. In certain aspects, user device201 may be configured to receive indications of one or more targetpartitions. For example, user device 201 may be configured to receiveindications of one or more target partitions from one or more of a user,a non-transitory memory, and a remote computer. In some aspects, userdevice 201 may be configured to receive indications of a mappingfunction and/or reduction function. For example, user device 201 may beconfigured to receive indications of at least one mapping functionand/or at least one reduction function from one or more of a user, anon-transitory memory, and a remote computer. In some aspects, userdevice 201 may be configured to receive indications of one or moreuniform partitions corresponding to the one or more target partitions.For example, user device 201 may be configured to receive theindications of uniform partitions from one or more of a user, anon-transitory memory, and a remote computer. In certain aspects, userdevice 201 may be configured to receive indications of one or morestatistical measures corresponding to the desired target partitions. Forexample, user device 201 may be configured to receive indications of theone or more statistical measures from a user, a non-transitory memory,datasource 203, cluster 205, and a remote computer. Based on thereceived indications of the one or more statistical measures, userdevice 201 may be configured to generate one or more uniform partitionscorresponding to the one or more desired target partitions.

Cluster 205 may be configured to create the calculation partition,consistent with disclosed embodiments. In some aspects, cluster 205 maybe configured to receive analysis information from user device 201.Analysis information may comprise one or more of a mapping function,reduction function, target partitions, uniform partitions, andstatistical measures. In some aspects, cluster 205 may be configured todetermine one or more statistical measures from data elements receivedfrom datasource 203. In certain aspects, cluster 205 may be configuredto determine one or more uniform partitions based on one or morereceived and/or determined statistical measures. In some aspects,cluster 205 may be configured to determine the calculation partitionbased on the one or more target partitions and the one or more receivedand/or determined uniform partitions. As described above, cluster 205may comprise worker nodes. The worker nodes may be configured asmapper(s) 213 and reducer(s) 217. In some embodiments, controller 211may configure mapper(s) 213 and reducer(s) 217. In certain embodiments,user device 201 may configure mapper(s) 213 and reducer(s) 217. Invarious embodiments, controller 211 may enable reducer(s) 217 to obtainintermediate data 215 generated by mapper(s) 213. In some embodiments,controller 211 may shuffle, or route, intermediate data 215 toreducer(s) 217.

Dataset processing system 200 may be configured to process data elementsin step 503, consistent with disclosed embodiments. In some aspects,cluster 205 may be configured to process data elements received fromdatasource 203. For example, mapper(s) 213 may be configured to yieldintermediate data 215 comprising keys according to the mapping function.In some aspects, controller 211 may be configured to provide a mappingfunction implementing the calculation partition to mapper(s) 213. Incertain aspects, user device 201 may be configured to provide a mappingfunction implementing the calculation partition to mapper(s) 213.

Dataset processing system 200 may be configured to generate output data207 in step 505, consistent with disclosed embodiments. In some aspects,cluster 205 may be configured to reduce intermediate data 215. Forexample, reducer(s) 217 may be configured to retrieve groups ofintermediate data 215 corresponding to the keys. In some aspects,reducer(s) 217 may be configured to apply a reduction function to thegroups of intermediate data 215 to generate secondary values. In someembodiments, cluster 205 may be configured to store or stream thesecondary values as output data 207. In certain embodiments, reducer(s)217 may be configured to apply the reduction function, or anotherfunction, to groups of secondary values to generate analysis resultsaccording to the one or more target partitions. In some aspects, cluster205 may be configured to store or stream the analysis results as outputdata 207. As would be recognized by one of skill in the art, thedisclosed embodiments may perform multiple iterations of groupingresults and applying the reduction function, or another function, togenerate the analysis results from the intermediate data 215.

Dataset processing system 200 may be configured to provide analysisresults in step 505, consistent with disclosed embodiments. In someembodiments, display device 209 may be configured to retrieve or receivesecondary values from output data 207. In certain aspects, displaydevice 209 may be configured to apply a reduction function to groups ofintermediate values to generate analysis results according to the one ormore target partitions. In certain embodiments, display device 209 maybe configured to retrieve or receive analysis results from output data207. Display device 209 may be configured to provide analysis results,as described above with respect to FIG. 2. In certain embodiments,display device 209 and user device 201 may be the same device. In someembodiments, a user (not shown) may operate display device 209, ordirect operation of display device 209.

FIG. 6 depicts a schematic illustrating an exemplary component ofdataset processing system 200, consistent with disclosed embodiments. Insome embodiments, computing system 600 may include a processor 605,memory 610, and network adapter 625. In certain aspects, computingsystem 600 may include one or more of display 615 and I/O interface(s)620. These units may communicate with each other via bus 630, orwirelessly, and may reside in a single device or multiple devices.

Consistent with disclosed embodiments, processor 605 may comprise one ormore microprocessors, central processing units (CPUs), graphicalprocessing units (GPUs), application specific integrated circuits(ASICs), or Field Programmable Gate Arrays (FPGAs). Memory 610 mayinclude non-transitory memory containing non-transitory instructions,such as a computer hard disk, random access memory (RAM), removablestorage, or remote computer storage. In some aspects, memory 610 may beconfigured to store software programs. In some aspects, processor 605may be configured to execute non-transitory instructions and/or programsstored on memory 610 to configure computing system 600 to performoperations of the disclosed systems. In various aspects, as would berecognized by one of skill in the art, processor 605 may be configuredto execute non-transitory instructions and/or programs stored on aremote memory to perform operations of the disclosed systems. Display615 may be any device which provides a visual output, for example, acomputer monitor, an LCD screen, etc. I/O interfaces 620 may includemeans for communicating information to computer system 600 from a userof computing system 600, such as a keyboard, mouse, trackball, audioinput device, touch screen, infrared input interface, or similar device.Network adapter 625 may include components for enabling computing system600 to exchange information with external networks. For example, networkadapter 625 may include a wireless wide area network (WWAN) adapter, aBluetooth® module, a near field communication module, or a local areanetwork (LAN) adapter.

Other embodiments will be apparent to those skilled in the art fromconsideration of the specification and practice of the disclosedembodiments disclosed herein. It is intended that the specification andexamples be considered as exemplary only, with a true scope and spiritof the disclosed embodiments being indicated by the following claims.Furthermore, although aspects of the disclosed embodiments are describedas being associated with data stored in memory and other tangiblecomputer-readable storage mediums, one skilled in the art willappreciate that these aspects can also be stored on and executed frommany types of tangible computer-readable media, such as secondarystorage devices, like hard disks, floppy disks, or CD-ROM, or otherforms of RAM or ROM. As used herein, instructions may include, withoutlimitation, software or computer code, as would be understood by one ofskill in the art. The disclosed embodiments are not intended to belimited to instructions in a particular programming language or format.Accordingly, the disclosed embodiments are not limited to the abovedescribed examples, but are instead defined by the appended claims inlight of their full scope of equivalents.

Moreover, while illustrative embodiments have been described herein, thescope includes any and all embodiments having equivalent elements,modifications, omissions, combinations (e.g., of aspects across variousembodiments), adaptations or alterations based on the presentdisclosure. The elements in the claims are to be interpreted broadlybased on the language employed in the claims and not limited to examplesdescribed in the present specification or during the prosecution of theapplication, which examples are to be construed as non-exclusive.Further, the steps of the disclosed operations of the disclosed systemsfor parallel processing of datasets with dynamic skew compensation canbe modified in any manner, including by reordering, inserting, ordeleting steps. Furthermore, these disclosed operations, considered as asequence of steps, constitute methods for parallel processing ofdatasets with dynamic skew compensation. It is intended, therefore, thatthe specification and examples be considered as example only, with atrue scope and spirit being indicated by the following claims and theirfull scope of equivalents.

What is claimed is:
 1. A non-transitory computer readable medium storinga set of instructions that is executable by one or more processors of acontroller in a cluster which further comprises a plurality of workernodes, where the instructions cause the controller to perform a methodcomprising: configuring at least one of the worker nodes as a mapperwith a mapping function to process data elements received from a datasource into intermediate data comprising a plurality of key-value pairs,where the mapping function implements: a first combination partitionconfigured to divide the data elements into a first set of groupsdetermined by a union of a first target partition and a first uniformpartition for a first variable of the data elements, wherein the firsttarget partition divides the data elements into groups according topredetermined values or value ranges of the first variable, and thefirst uniform partition divides the data elements into similarly sizedgroups based on a statistical distribution of the values of the firstvariable, and a second combination partition configured to divide thedata elements into a second set of groups determined by a union of asecond target partition and a second uniform partition for a secondvariable of the data elements, wherein the second target partitiondivides the data elements into groups according to predetermined valuesor value ranges of the second variable, and the second uniform partitiondivides the data elements into similarly sized groups based on astatistical distribution of the values of the second variable, and wherethe mapping function defines a unique key for each intersection betweenthe first set of groups and the second set of groups and each key in thekey-value pairs of the intermediate data corresponds to one of theunique keys; and configuring a plurality of the worker nodes as reducersto process the intermediate data into output data for a display deviceor for subsequent processing, where each reducer processes the key valuepairs corresponding to one or more of the unique keys and all of thekey-value pairs which share the same unique key will be processed byexactly one reducer.
 2. The non-transitory computer readable medium ofclaim 1, wherein the union combines the first target partition of thefirst variable and the first uniform partition of the first variable. 3.The non-transitory computer readable medium of claim 1, wherein thestatistical distribution of the values of the first variable isdetermined using a plurality of statistical measures of the firstvariable values.
 4. The non-transitory computer readable medium of claim3, wherein the method further comprises determining the plurality ofstatistical measures.
 5. The non-transitory computer readable medium ofclaim 3, wherein the statistical distribution of the first variablevalues is a probability distribution function of the first variablevalues.
 6. The non-transitory computer readable medium of claim 3,wherein the first variable is a numeric variable and the plurality ofstatistical measures of the first variable values are either quantilesor estimated quantiles of the first variable values.
 7. Thenon-transitory computer readable medium of claim 1, wherein the firstuniform partition uses a random or pseudorandom value to divide the dataelements into the similarly sized groups.
 8. The non-transitory computerreadable medium of claim 1, wherein the first uniform partition is basedon a range of values selected to optimize system processing speed. 9.The non-transitory computer readable medium of claim 1, wherein themethod further comprises receiving the predetermined values or valueranges of the first variable of the first target partition from a userdevice.
 10. The non-transitory computer readable medium of claim 1,wherein the method further comprises determining the first uniformpartition.
 11. The non-transitory computer readable medium of claim 1,wherein the data elements comprise the first variable values.
 12. Acluster for parallel processing of datasets with dynamic skewcompensation, comprising: at least one mapper worker node comprising aprocessor and a storage medium comprising instructions that, whenexecuted by the processor, cause the mapper worker node to process dataelements received from a data source into intermediate data comprising aplurality of key-value pairs using a mapping function; a plurality ofreducer worker nodes each comprising a processor and a storage mediumcomprising instructions that, when executed by the processor, cause thereducer worker node to process the intermediate data into output datafor a display device or for subsequent processing, where each reducerprocesses the key-value pairs corresponding to one or more of the uniquekeys and all of the key-value pairs which share the same unique key willbe processed by exactly one reducer; and a controller comprising aprocessor and a storage medium comprising instructions that, whenexecuted by the processor, cause the controller to configure the mappingfunction to implement: a first combination partition configured todivide the data elements into a first set of groups determined by aunion of a first target partition and a first uniform partition for afirst variable of the data elements, wherein the first target partitiondivides the data elements into groups according to predetermined valuesor value ranges of the first variable, and the first uniform partitiondivides the data elements into similarly sized groups based on astatistical distribution of the values of the first variable, and asecond combination partition configured to divide the data elements intoa second set of groups determined by a union of a second targetpartition and a second uniform partition for a second variable of thedata elements, wherein the second target partition divides the dataelements into groups according to predetermined values or value rangesof the second variable, and the second uniform partition divides thedata elements into similarly sized groups based on a statisticaldistribution of the values of the second variable, and wherein theconfigured mapping function defines a unique key for each intersectionbetween the first set of groups and the second set of groups and eachkey in the key-value pairs of the intermediate data corresponds to oneof the unique keys.
 13. The cluster of claim 12, wherein the firstuniform partition divides the data elements into similarly sized groupsby establishing a maximum group size determined from the statisticaldistribution of the values of the first variable.
 14. The cluster ofclaim 12, wherein the instructions of the controller further compriseinstructions to determine a plurality of statistical measures of thefirst variable values and the statistical distribution of the values ofthe first variable is determined using the plurality of statisticalmeasures.
 15. The cluster of claim 14, wherein the statisticaldistribution of the first variable values is a probability distributionfunction of the first variable values.
 16. The cluster of claim 12,wherein the predetermined values or value ranges of the first variableof the first target partition are received by the controller from a userdevice.
 17. A system for parallel processing of datasets with dynamicskew compensation in a cluster comprising a plurality of worker nodes,comprising: a controller comprising a processor and a non-transitorystorage medium comprising instructions that, when executed by theprocessor, cause the controller to: configure at least one of the workernodes as a mapper with a mapping function to process data elementsreceived from a data source into intermediate data comprising aplurality of key-value pairs, where the mapping function implements: afirst combination partition configured to divide the data elements intoa first set of groups determined by a union of a first target partitionand a first uniform partition for a first variable of the data elements,wherein the first target partition divides the data elements into groupsaccording to predetermined values or value ranges of the first variable,and the first uniform partition divides the data elements into similarlysized groups based on a statistical distribution of the values of thefirst variable, and a second combination partition configured to dividethe data elements into a second set of groups determined by a union of asecond target partition and a second uniform partition for a secondvariable of the data elements, wherein the second target partitiondivides the data elements into groups according to predetermined valuesor value ranges of the second variable, and the second uniform partitiondivides the data elements into similarly sized groups based on astatistical distribution of the values of the second variable, and wherethe mapping function defines a unique key for each intersection betweenthe first set of groups and the second set of groups and each key in thekey-value pairs of the intermediate data corresponds to one of theunique keys; and configure a plurality of the worker nodes as reducersto process the intermediate data into output data for a display deviceor for subsequent processing, where each reducer processes the key-valuepairs corresponding to one or more of the unique keys and all of thekey-value pairs which share the same unique key will be processed byexactly one reducer.
 18. The system of claim 17, wherein the firstuniform partition divides the data elements into similarly sized groupsby establishing a maximum group size determined from the statisticaldistribution of the values of the first variable.
 19. The system ofclaim 17, wherein the instructions of the controller further compriseinstructions to determine a plurality of statistical measures of thefirst variable values and the statistical distribution of the values ofthe first variable is determined using the plurality of statisticalmeasures.
 20. The system of claim 17, further comprising a user devicehaving a processor and a storage medium comprising instructions that,when executed by the processor, cause the user device to provide thepredetermined values or value ranges of the first variable of the firsttarget partition to the controller.